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ABSTRACT 



The Third International Mathematics and Science Study 
(TIMSS) is the largest, most comprehensive, and most rigorous international 
comparison of education ever undertaken. TIMSS findings show similarities and 
differences in the processes and outcomes of schooling between the United 
States and East Asian countries, particularly Japan. This report reviews the 
initial findings from the TIMSS eighth-grade report, with a special focus on 
the United States and Japan, and discusses missing or incomplete links in 
international education studies. The initial findings highlight similarities 
between students’ learning experiences at home and critical differences 
between those at school. These patterns imply that Japanese and U.S. students 
do not differ in out-of -school learning experiences, but there are 
differences neglected in TIMSS highlights, such as private tutoring. On the 
other hand, TIMSS researchers found substantial differences in the content 
and process of instruction at school and differences in instructional 
organization. While initial TIMSS findings show that the United States is 
still far from achieving the national goal of being first in the world in 
mathematics and science, these aggregate national patterns ignore the 
enormous local variation in the U.S. It is not clear whether federated 
countries like the United States are really comparable to other countries . 

For a valid comparison of the United States with highly centralized East 
Asian countries, it is suggested that the American states be treated as 
comparable national units. For a reliable comparison of the countries over 
time, it is suggested that current practices and outcomes be compared to past 
counterparts, something that will take into account the possibility of 
educational convergence between the United States and East Asian countries. 

To make fair, valid, and reliable comparisons, future international education 
studies need to take into account the linkages between formal and informal 
learning, consider local variation within a country, and compare current and 
past practices and outcomes. (Contains 1 figure, 1 table, and 33 references.) 
(SLD) 
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Introduction 



The Third International Mathematics and Science Study (TIMSS) is the 
largest, most comprehensive, and most rigorous international comparison of 
education ever undertaken. The TIMSS results show us where U.S. education 
stands not only in terms of academic achievement, but also in terms of the 
curriculum and instruction that students receive (See Beaton et al., 1996a, 
1996b; Howson, 1995; Schmidt et al., 1996). Indeed, Pursuing Excellence, a 
series of reports published by the National Center for Education Statistics 
provide a rich synthesis of the findings from TIMSS on U.S. fourth-grade, 
eighth-grade, and twelfth-grade mathematics and science education, and thus 
has attracted nationwide media coverage and public attention. 

The TIMSS findings show similarities and differences in the processes 
and outcomes of schooling between the U.S. and East Asian countries, 
particularly Japan. On the one hand, American education policy over the last 
decade seemed to follow the East Asian schooling model that fulfills higher 
and tougher education standards for all students (see Carnegie Forum on 
Education and the Economy, 1986; National Commission on Excellence in 
Education, 1983). On the other hand, researchers explained the learning gap 
between the U.S. and East Asian countries in terms of the different 
institutional or cultural conditions that support the systemic implementation 
of education standards in each country. This includes differences in (1) 
curricular governance systems; and (2) parental values and student attitudes 
(see Stevenson & Baker, 1991; Stevenson & Stigler, 1992). 

While large-scale international studies such as the TIMSS are not 
designed to address particular issues in individual countries, they often have 
far-reaching implications for national education policy and research beyond 
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the original scope of analysis. For the comparison of the U.S. with Japan or 
East Asian countries in general, the initial findings from the TIMSS show 
that some important linkages were neglected. First, the TIMSS focuses on 
formal education without paying attention to the linkages between schooling 
and out-of-school learning experiences. Second, the TIMSS reports aggregate 
national patterns while neglecting local variation. Third, the TIMSS focuses 
on the current status of educational practices and outcomes without paying 
attention to linkages with past practices and outcomes. In light of these 
concerns, I will review the initial findings from TIMSS 8th grade report 
(NCES, 1996) with special focus on the cases of the U.S. and Japan, and discuss 
missing or incomplete links in international education studies. 



Linkages between formal education and out-of-school learning 

The initial findings from the TIMSS analysis of the U.S. and Japan 
highlight similarities between students' learning experiences at home but 
critical differences between those at school. On the one hand, the TIMSS 
researchers found that students in both countries spend a relatively 
comparable amount of time on self-study and in leisure activities after school. 
First, Japanese and U.S. students spend between 30 minutes and an hour 
studying math outside of school, and about the same amount studying 
science. Second, heavy TV watching is as common among U.S. eighth graders 
as it is among their Japanese counterparts. 

These patterns imply that Japanese students do not differ from U.S. 
counterparts in out-of-school learning experiences. However, there are 
differences neglected in the TIMSS highlights. After-school private tutoring 
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practices (juku) that complement formal schooling or prepare students for 
high-stakes exams are pervasive in Japan (see Goya, 1993; Harnish, 1994; 
Rohlen, 1980). This is also the case for many other Asian nations that 
outperform the U.S. in mathematics and science, including Korea, Singapore, 
and Taiwan. A national survey of Korean middle/high school students 
shows that the average student receives about 7 hours of tutoring services 
every week (Shin et al., 1991).! As shown in Table 1, the survey also shows 
that the perceived effectiveness of private tutoring is overwhelming and that 
this perception motivates students to undertake private tutoring. 

Table 1. Frequency and Percentage of Korean Students Reporting the 
Effectiveness of Private Tutoring 



How Much Help Do (Did) You Receive from Private 
Tutoring in Improving Your Academic Achievement 

(College Entrance)? 

No /Little Some Great 

N (Row %) N (Row %) N (Row %) Total 



Type of Middle School 59 (4.9) 665 (54.7) 492 (40.5) 1216 

Survey High School 51 (10.7) 306 (62.2) 133 (27.0) 460 

Respondents College Freshman 75 (18.9) 226 (57.0) 96 (24.2) 397 



According to the NCES report, sixty- four percent of Japanese eighth 
graders reported attending weekly extra lessons in math, and 41 percent in 
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science (NCES, 1996). In contrast, a quarter of the students in the U.S. 
reported that they worked at a paid job before or after school. Consequently, 
the exclusive focus on students' reports about their learning experiences at 
home obscures substantial differences in the type of additional learning 
opportunities available to students. 

On the other hand, the TIMSS researchers found substantial differences 
in the content and process of instruction at school. The content taught in U.S. 
eighth-grade math classrooms is at a seventh-grade level in comparison with 
Japan. Moreover, the pedagogy in the U.S. classrooms also differed 
significantly from that of Japanese classrooms. American teachers emphasizes 
skill acquisition, whereas Japanese teachers focus on understanding. When 
we shift our attention to out-of-school learning, classroom pedagogy in Japan 
is seen in a different light. As the NCES report notes, private tutoring in 
Japan focuses on the review and practice of basic skills (NCES, 1996). This dual 
system of learning assists slower students who need to review prior material, 
and provides all students with extra practice on concepts taught in the 
classroom. 

The TIMSS researchers also note differences in instructional 
organization. In the U.S., eighth-grade students of different abilities are 
typically divided into different classrooms, whereas there is no ability 
grouping in Japan at this grade level. However, this contrast ignores the fact 
that juku sorts Japanese students into ability groups and provides differential 
curriculum. In fact, the content of the courses provided at the juku ranges 
from remedial to highly accelerated (White, 1987). Further, the opportunity to 
take advantage of private lessons differ among students from different 
socioeconomic backgrounds. Thus, juku, or private after-school class tutoring, 
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reinforces the principle of competitive selection while egalitarianism is 
upheld through the formal schooling process. 
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Linkages between aggregate national pattern and local variation 

Initial findings from TIMSS on U.S. eighth-grade mathematics and 
science education show that the U.S is still far from achieving the national 
goal of being first in the world in mathematics and science achievement by 
the year 2000 (NCES, 1996). In TIMSS mathematics, the U.S. eighth graders 
scored below the international average. This result, however, may not 
surprise people who are acquainted with previous international surveys of 
achievement such as the International Association for the Evaluation of 
Educational Achievement (IEA) in the 1960s and 1980s, and the International 
Assessment of Educational Progress (IAEP) in the early 1990s that showed 
basically the same results (see Raizen and Jones, 1985; McKnight et al., 1987; 
Educational Testing Service, 1989). 

However, these aggregate national patterns ignore enormous local 
variation in the U.S. Most American states are not only comparable to many 
countries in size or population, but each state is also responsible for its own 
educational system, similar to the way in which most other national 
governments are responsible for the educational system. In fact, previous 
comparison of individual states with other nations showed that not all U.S. 
school systems are alike, and that differences in achievement exist among the 
American states that are as great as the differences among the countries 
examined (see Beaton & Gonzalez, 1993; Salganik et al., 1993).^ Figure 1 shows 




Figure 1. Mathematics proficiency scores for 13-year-olds in countries and 
public school 8th grade students in states, by country (1991) and state (1992) 




Level 200: Beginning skills and understandings. 

Level 250: Numerical operations and beginning problem solving. 
Level 300: Moderately complex procedures and reasoning 



Source: Educational Testing Service, IAEP/NAEP Cross-linking Study, 1993; U.S. Department 
of Education, National Center for Education Statistics, NAEP 1992 Mathematics Report Card 
for the Nation and the States, 1993. 
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that the highest performing states in the U.S. (e.g., Iowa, North Dakota, and 
Minnesota) perform as well as the highest performing countries (e.g., Taiwan 
and Korea), whereas the lowest performing states in the U.S. perform as 
poorly as the lowest performing countries in the world. In addition to 
national assessment, the U.S. TIMSS offered states the opportunity to assess a 
state-representative sample of their students. However, the state participation 
rate was very low because individual states were responsible for all costs 
associated with State TIMSS: Colorado, Illinois, and Minnesota joined in this 
program. 

The term curriculum framework has recently been used in national 
and international studies aimed at a broad analysis of curriculum across 
schools, districts, states or countries. The National Assessment of Educational 
Progress (NAEP) is now written and conducted based on a content framework 
that is developed by a consensus panel of subject experts, educators, and 
researchers (CCSSO, 1991; National Assessment Governing Board, 1994). At 
the same time, the Third International Mathematics and Science Study 
(TIMSS) is based on mathematics and science frameworks developed by an 
international consensus among scientists, mathematicians, educators, and 
researchers from participating countries (Robitaile et al., 1993). 

Notwithstanding a growing national and international consensus 
around curriculum and instruction in core academic subjects, it remains 
dubious whether federated countries like the U.S. are comparable to others. 
The TIMSS researchers found that the U.S. curriculum is less focused and less 
advanced than the Japanese curriculum. They also found that lessons with 
high-quality mathematical reasoning are a rare phenomenon in the U.S. 
These national aggregate patterns, however, may obscure the fact that in the 
U.S., states play independent roles in developing and implementing 
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curriculum. With the publication of the National Council of Teachers of 
Mathematics (NCTM) Curriculum and Evaluation Standards for School 
Mathematics (1989), many states worked on ways to represent the Standards 
in state curriculum frameworks (Blank and Dalkilic, 1992). A recent study 
found that the substance of such newly developed or revised curriculum 
frameworks vary widely among states (Blank and Pechman, 1995). Thus, the 
state as the unit of analysis seems to be more appropriate than the nation 
when examining curriculum. 



Linkages between current and past educational practices and outcomes 

A rough comparison of TIMSS results with previous IEA/IAEP results 
indicates that U.S. eighth graders' overall standing in math has hardly 
improved relative to their international counterparts (NCES, 1996). Despite 
the seemingly persistent learning gap between the U.S. and other industrial 
countries, reliable international comparisons over time are difficult because 
the assessments are based on different instruments and populations. 
Achievement in TIMSS was initially intended to be linked with the results of 
two earlier IEA studies, that is, the Second International Mathematics Study 
(SIMS) and the Second International Science Study (SISS). However, formal 
links between TIMSS and SIMS were never realized because the target 
populations were not equivalent (Garden & Orpwood, 1996). Consequently, 
the lack of linkages between the TIMSS and its antecedents deprived 
researchers of the opportunity to systematically examine whether there have 
been significant changes in educational processes and outcomes during the 
interim period. 
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While cross-sectional studies using large-scale international databases 
have been concerned primarily with the comparison of current educational 
practices and outcomes, another branch of international studies focused on 
institutional changes across national school systems. This research observed a 
global convergence in both educational ideology and educational structure 
(see Meyer et al., 1979; Ramirez and Boli, 1987). For example, many countries 
have addressed similar issues using different labels such as educational 
excellence, productivity, efficiency, and accountability. Additionally, global 
school reform over the last decade seem to have been shaped by two sets of 
forces. One is growing public distrust of school bureaucracy in a climate of 
rapid political change (Wong, 1994). The other is growing international 
competition in the context of the global economy (Kearns and Doyle, 1991). 

The consequences of these factors for education policies have varied 
between countries according to their educational conditions and problems. 
Indeed, the U.S. and Japan took quite contrasting approach to school reform 
during the 1980s and early 1990s. In the U.S. where low educational 
expectations and achievement were identified as central problems, efforts 
were made to set higher education standards and require stronger school 
accountability (see Ravitch, 1995). These policies, which have emerged since 
the 1983 national report A Nation at Risk, culminated with the 1989 national 
education goals (enacted into the Goals 2000 in 1994). In contrast, Japan 
identified the problem as uniform control and excessive competition, 
resulting in the lack of a humane education. To address this issue, the 
National Commission on Educational Reform, established by the 
government in 1984, made efforts to individualize the curriculum and 
diversify the schools (see Lincicombe, 1993). 
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Assuming that national reform policies promote systemic change, one 
may hypothesize that the past gap in classroom learning opportunities 
between the two industrial countries has narrowed as a result of their 
different reform paths. For example, increased instructional time and 
homework assignment was likely to be promoted in the U.S., whereas the 
opposite was likely to happen in Japan. In this regard, some of the TIMSS 
results that appear contrary to popular beliefs are noteworthy. 



Figure 2. Hours of Mathematics Instructional Time Per Year for Eighth- 
Graders based on the SIMS (1982) and TIMSS (1995) Reports 




■ United States 
CD Japan 



1 982 



1 995 



Note. The 1982 SIMS results involve eighth grade in U.S. but seventh grade in Japan. 



First, U.S. eighth graders tend to spend more hours per year in math 
and science classes than Japanese students. Nevertheless, the previous 
comparison based on the SIMS results (McNight et al., 1987) showed that the 
U.S. exceeded Japan in terms of instructional time in mathematics even back 
in 1982 (see Figure 2). Thus, there is no evidence that as a result of national 
school reforms Japanese students get assigned relatively small amount of 
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instructional time on math while American students get more of math 
instruction. 

Second, Japanese teachers tend to assign less homework than U.S. 
teachers. ^ Having found that Japanese teachers assigned less homework than 
U.S. teachers, but at the same time Japanese students reported that they 
studied about as much as their U.S. counterparts, the authors of Pursuing 
Excellence ask how Japanese students were motivated and supported in this 
extra work and seem to seek its answer in the exigency of high school 
entrance examinations for Japanese 8th graders (NCES, 1996). By linking these 
and other TIMSS results to their counterparts from previous international 
studies, we may be able to explore the potential of educational changes that 
are related to school reform policies. 



Conclusion 



The initial findings from TIMSS strongly suggest that substantial 
differences in the quality of curriculum and instruction account for the 
learning gap between the U.S. and East Asian countries, particularly Japan. 
Despite the relevance of school-related findings to international comparisons, 
the exclusive focus on formal schooling may ignore the supplementary 
function of informal education. For a fairer comparison of U.S. and East 
Asian education, I suggest a closer look at the role of after-school private 
tutoring practices in East Asian countries. At the same time, the exclusive 
attention to national comparisons may obscure the size of local variation in 
different countries. For a valid comparison of the U.S. with highly centralized 
East Asian countries, I suggest the American states be treated as comparable 
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national units. Finally, the exclusive concern with the current status of 
educational practices and outcomes may detract our attention from an 
examination of dynamic changes in each country. For a reliable comparison 
of the countries over time, I suggest a comparison of the current practices and 
outcomes to the past counterparts. This will take into account the possibility 
of educational convergence between the U.S. and East Asian countries that 
result from their quite different reform paths. 

Indeed, these problems are not unique to the U.S. and East Asian 
countries in TIMSS analysis. In an attempt to make data collection more 
efficient or to make research findings more generalizable, any large-scale 
international assessment studies are likely to introduce certain biases in their 
research agenda. To insure fair, valid, and reliable comparisons, future 
international education studies need to take into account the linkages 
between formal and informal learning, consider local variation within a 
country, and compare current with past practices and outcomes. In the 
meantime, researchers who analyze large-scale international assessment data 
as well as the policy community who utilize the research findings need to be 
more aware of the limitations of the currently available information. 
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Notes 



1 3 



1 It is also noteworthy that private expenditures on tutoring is often greater 
than public school expenditures, despite the fact that students spend much 
more time on school learning than on tutoring: Last year, Koreans spent $25 
billion or fully 150 % of the government's education budget on private 
tutoring (Asia Week, 1997). 

2 This comparison was made possible through an experimental linking of 
state-by-state assessment data (NAEP TSA) with international assessment data 
(IAEP). However, these data bases are not aligned with each other to allow for 
the formal equating and rigorous comparisons between the U.S. states and the 
IAEP nations (see Linn & Baker, 1995). 

3 86 percent of U.S. mathematics teachers assigned 3 to 5 times per week in 
comparison to 21 percent of Japanese teachers. But the typical amount of 
homework assignment was about thirty minutes or less in both countries. 
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